Probability and Statistics in Computational Linguistics: a brief review

Computational linguistics studies the computational processes involved in language learning, production, and comprehension. Computational linguists believe that the essence of these processes (in humans and machines) is a computational manipulation of information. Computational psycholinguistics studies psychological aspects of human language (e.g., the time course of sentence comprehension) in terms of such computational processes. Natural language processing is the use of computers for processing natural language text or speech. Machine translation (the automatic translation of text or speech from one language to another) began with the very earliest computers (Kay et al., 1994). Natural language interfaces permit computers to interact with humans using natural language, e.g., to query databases. Coupled with speech recognition and speech synthesis, these capabilities will become more important with the growing popularity of portable computers that lack keyboards and large display screens. Other applications include spell and grammar checking and document summarization. Applications outside of natural language include compilers, which translate source code into lower-level machine code, and computer vision (Foo, 1974, 1982). The notion of a grammar is central to most work in computational linguistics and natural language processing. A grammar is a description of a language; usually it identifies the sentences of the language and provides